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ARLY in the study of intermittent 
reinforcement (3, 5, 6, 12) one con- 
clusion emerged as a general Jaw of learn- 
ing: schedules of intermittent reiniorce- 
ment result in greater resistance to ex- 
tinction than do comparable schedules 
of continuous reinforcement. 

Perhaps the most immediate conse- 
quence of this finding was that the then- 
current formulations of the law of ef- 
fect had to be revised. Until the advent 
of intermittent reinforcement, the law of 
effect usually took something like the 
following form: habit strength and, 
therefore, its measure, resistance to ex- 
tinction, are increasing functions of the 
number of reinforced responses. ‘This 
supposition was untenable after the stud- 
ies of intermittent reinforcement had be- 
come well known. 

The theoretical attempts (1, 4, 9, 11) 
to rescue the law of effect were very 
nearly as numerous as the experimental 
studies which had placed the law in 
jeopardy. Nevertheless, a single explana- 
tion gradually became general: in inter- 
mittent reinforcement the acquisition 
conditions, since they involve a sizable 
number of extinction trials, are more 
similar to extinction conditions than are 
the acquisition conditions in continuous 
reinforcement. Hence, there is more 
stimulus generalization from acquisition 

*This monograph constitutes tle essence of 
a doctoral dissertation done at the University of 
California at Los Angeles under th direction of 
Professor Howard C. Gilhousen, for whose en- 


couragement and criticism the author is deeply 
grateful. 


to extinction in intermittent-reward 
training than in continuous-reward train- 
ing. This greater stimulus generalization 
serves to perpetuate the intermittently 
acquired response alter the continuously 
acquired response has extinguished. ‘This 
theory, though derived from outright 
studies of intermittent reinforcement, 
has consequences which go far beyond 
its historical origins. The principal pur- 
pose of the present study is to test three 
of these more remote consequences. 
Altogether there are in the present 
study four groups of experimental ani- 
mals: escape, avoidance, intermittent es- 
cape, and limited avoidance. The ex- 
tinction conditions for all four groups 
are the same; the acquisition conditions, 
however, differ. More specifically, the 
acquisition conditions for the three 
groups of nonescape animals are all more 
similar to extinction conditions than are 
the acquisition conditions for the escape 
animals. Whether the acquisition con- 
ditions for the three nonescape groups 
differ among themselves in terms of sim- 
ilarity we cannot say. All we can assert 
with confidence is that all three are more 
similar to extinction conditions than are 
the acquisition conditions for the escape 
animals. In conseqence, stimulus general- 
ization from acquisition to extinction 
should be greater among the animals of 
the three nonescape groups. From this 
we may conclude that the avoidance, 
intermittent-escape, and limited-avoid- 
ance animals should all be more resistant 
to extinction than the escape animals. 
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Of these three consequences, the first, 
that resistance to extinction should be 
greater among the avoidance than among 
the escape animals, has already been 
tested and confirmed by Sheffield and 
Temmer (10), who used a single, total- 
response measure. We, on the other 
hand, have broken up the total response 
into a starting and a running response. 
We are, therefore, in a position to ask 


The apparatus employed in the present study 
was a grid runway leading to a wooden goal 
compartment. The grid measured 48 inches in 
length, 7 inches in width, and was surrounded 
on all sides, except for the escape “hole,” by 
18-inch-high walls. The escape hole, 6 inches in 
height, was cut in the wall at the far end of 
the grid. The hole led to the goal box, which 
was 1 foot deep, 7 inches wide, like the runway, 
and surrounded by 6-inch-high walls. The en- 
tire apparatus was made of plywood and was 
unpainted except for the goal box, which was 
painted black. A guillotine door at the entrance 
to the goal box prevented retracing. 

One foot from the beginning of the grid two 
round holes were cut opposite each other in 
the side walls. These holes permitted a beam 
to an eiectric eye to pass over the runway. Just 
inside the goal box a second pair of holes per- 
mitted a second beam to pass over the runway. 
These two beams provided our response meas- 
ures. When the animals were dropped on the 
grid, two electric clocks, calibrated in hun- 
dredths of a second, were activated. When the 
rat cut the first beam, the first clock stopped; 
when it cut the second beam, the second clock 
stopped. Hence, our two raw measures were 
starting time and total time. 

The shock stimulator delivered 0.25 milli- 
amperes to the grid. Since the resistance of the 
rat varies between 200 and 400 thousand ohms, 
the voltage drop across the shocked rat varied 
between 50 and 100 volts. These amperages and 


The subjects were 44 male, hooded 
rats. All the animals were born within a 
month of each other and all were ap- 
proximately 200 days old at the start of 
the experiment. All had participated two 
months earlier in a thirst-motivated, bar- 
pressing experiment. None of the ani- 
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I. APPARATUS 


Il. SUBJECTS AND PROCEDURE 


the question: Does the starting or the 
running response undergo the greater de- 
terioration in extinction? The answer to 
this question is given in the sections on 
extinction. 

The theoretical issues underlying the 
sections on recovery and acquisition will 
be presented together with the results to 
which they relate. 


voltages are approximately standard in shock 
situations. 

Connected with the shock stimulator was an 
intervalometer, which could be set so that at 
any desired delay after the clocks were activated 
shock would begin. In order to draw sufficient 
current through the relays that stopped the 
clocks, two light bulbs, used purely as resistors, 
had to be inserted in the circuit. 

In Sheffield and Temmer’s work the timing 
apparatus was silent. Ours was not. More 
specifically, two clicks were audible: the first 
sounded when the clocks were activated, and the 
second when the second beam was cut. The first 
click, since it coincided with dropping on the 
grid, was simply another cue to which the start- 
ing response could become conditioned. It, to- 
gether with dropping and the grid compartment, 
constituted the stimulus complex that _ ini- 
tiated each trial. The second click, since it co- 
incided with entrance into the goal box, con- 
stituted, together with the goal box itself, the 
stimulus complex that ended the trial. While 
these clicks do not affect the theory of the ex- 
periment, they may very well have influenced: 
the absolute levels of resistance to extinction ob- 
served. Sheffield and Temmer’s rats showed less 
resistance to extinction than did ours. One 
plausible explanation for this difference is that 
the clicks of our apparatus provided a more 
stable stimulus configuration to serve, on the 
one hand, as the conditioned stimulus, and, on 
the other, as secondary reward. 


mals had had any previous experience of 
shock. The animals varied in weight 
from 270 to 417 grams. Prior to the ex- 
periment the animals were weighed and 
ranked in order of increasing weight. 
The animals were then divided into sets 
of four by weight, beginning with the 
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four lightest animals and ending with 
the four heaviest. From each set one ani- 
mal was assigned at random to each 
group. All four groups were, therefore, 
paired for weight. 


Throughout the experiment the animals were 
maintained on an ad libitum schedule for both 
food and water. The experimental sessions were 
begun daily around noon. Two rats were run 
in each session. For the first it days one rat 
of the pair came from the escape group, and 
the second rat of the pair was his weight 
match from the limited-avoidance group. In the 
second 11 days one rat of the pair came from 
the avoidance group, while the second was its 
weight match the intermittent-escape 
group. 

There were certain procedural rules which 
applied equally to all groups. Fach rat was 
placed on the grid every 2 minutes. This inter- 
trial interval continued without disturbance from 
the beginning of acquisition until the animal 
was discontinued. Since the rats were run in pairs, 
one rat of the pair was placed on the grid every 
minute. All animals received 40 acquisition trials 
of their respective kinds, and then shock was 
withheld altogether. Each animal was run at 
least 10 trials in extinction. If the rat had not 
reached the criterion of three consecutive trials 
of 5 seconds or more in the first 10 trials of 
extinction, he was continued until he did or un- 
ul he passed the maximum permitted. The 
maximum fell at 60 trials in extinction. If an 
animal was still running at the end of 60 ex- 
tinction trials, he was discontinued. At no time, 
in acquisition or extinction, was an animal al- 
lowed to remain on the grid more than 10 sec- 
onds without disturbance. At 10 seconds he was 
gently paddled into the goal box. These maxima, 
6o trials in extinction and 10 seconds in any one 
trial, had to be imposed for purely practical 
reasons. In each trial two measures were re- 
corded: starting time, from dropping to the 
first beam; and running time, from the first to 
the second beam. Occasionally a rat would leap 
the first beam and its starting time for that 
trial would be lost. Whenever the occasional loss 
of starting time is of relevance to the presenta- 
tion of results, the treatment of the lost trials 
will be discussed. 

The escape animals in acquisition always 
landed on a charged grid. Since the grid re- 
mained charged, their only escape was through 
the escape hole into the goal box. At the 41st 
trial, shock was turned off and the trials con- 
tinued as before until the rat was discontinued 
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by criterion or by maximum. 

The avoidance animals landed on an un- 
charged grid. Two seconds later, if they had not 
reached the goal box, shock set in. Through- 
out the 4o-trial acquisition period the inter- 
valometer was set at 2 seconds. At the qist trial 
shock was permanently withheld and the animal 
was run until discontinued by criterion or by 
maximum, One thing is quite clear: the acquisi- 
tion conditions for the avoidance animal were 
considerably more similar to extinction condi- 
tions than were the acquisition conditions for 
the escape animal. In the first place, the avoid- 
ance animal on many an acquisition trial 
avoided shock altogether. When it did, the 
trial was, for all intents and purposes, identical 
to an extinction trial, since no shock was ex- 
perienced. In the second place, when the avoid- 
ance animal did receive shock, it received it 
from an initially uncharged grid. For the avoid- 
ance animal the grid was always uncharged ini- 
tially. In contrast, the escape animal never ex- 
perienced in acquisition a trial free from shock 
or an initially uncharged grid. 

The intermittent-escape animals, when they 
received shock, received it in the same way as 
did the escape animals, i.e., from the moment of 
landing. On their nonshock trials the grid was 
initially and permanenily uncharged. As acquisi- 
tion progressed, the shock trials became increas- 
ingly less frequent. Shock was presented on trials 
1-5, 7 10, 14, 19, 25, 32, and go. At the gist 
trial shock was turned off and the animals were 
run until discontinued by criterion or by maxi- 
mum, As we have seen, the greater similarity of 
avoidance training consists essentially of two 
elements: acquisition trials entirely free from 
shock, and receipt of shock, when experienced 
at all, from an initially uncharged grid. Clearly, 
the second element of similarity in avoidance 
training, receipt of shock from an initially un- 
charged grid, is altogether absent. The first ele- 
ment, however, interspersal in acquisition of 
shock-free trials, is decidedly present. 

For the limited-avoidance animal shock began 
0.8 second after his landing on the grid. This 
interval is too short for the rat to avoid shock 
altogether; it simply can’t cross the grid that 
fast. For this reason the first element of simi- 
larity in avoidance training, interspersal in ac- 
quisition of shock-free trials, is entirely ex- 
cluded. The second element, however, is present. 
The limited-avoidance animal always received 
shock from an initially uncharged grid. The 
limited-avoidance animals, like the rest, received 
40 acquisition trials, after which shock was turned 
off and the animals were run until discontinued 
by criterion or by maximum. 
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In treating the experimental results 
two techniques are employed: the Mann- 
Whitney U test (8) and the Wilcoxon 
test for paired scores (13). Both tests are 
nonparametric ranking techniques. Fur- 
thermore, both are designed to test the 
null hypothesis that there is “no mean 
difference between the two groups in- 
volved.” 

In evaluating our results we have used 
one-tailed tests exclusively. In justifica- 
tion of this usage we may say that we 
have always had theoretical and empiri- 
cal reason to expect the difference to 
lie in the observed direction. 

Although the Mann-Whitney test is designed 
for unpaired scores, it may be applied to paired 
data, In doing so, one arbitrarily fails to remove 
a source of variance which may result in non- 
significance when significance actually obtains. 
Whether or not one loses appreciable significance 
in applying the Mann-Whitney test to paired 
data depends upon the correlation, positive or 
negative, between the paired scores. If there is 
very little or no correlation, the loss is negligible. 


There are in the present study two types of 
pairing, by weight and by animal identity. In 
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some instances the Mann-Whitney test had to be 
applied to data which were initially paired for 
weight, because one or more animals of one of 
the groups had to be discarded. We have, there- 
fore, for the sake of uniformity, treated all 
data paired only by weight with the Mann- 
Whitney test. Comparison, when possible, with 
the Wilcoxon test shows negligible loss of sig- 
nificance. However, with scores paired on the 
basis of the animal which produced them, the 
Wilcoxon test has to be applied. Otherwise, the 
loss of significance is ruinous, 

When measures are used that have 
maximum values, for example, 60 trials 
in extinction, all animals which passed 
the maximum were given the same score 
and rank. We would, of course, have pre- 
ferred to do without maxima. However, 
granted their practical necessity, this con- 
vention seems to be the most reasonable 
treatment of the data. 

Since only two tests of significance are 
used, we do not specify the test used 
when quoting a significance level. How- 
ever, a glance at the appropriate table 
will quickly settle the matter. 


IV. EXTINCTION 


Escape vs. Avoidance Training 

Avoidance training presents in acqui- 
sition a set of conditions which are con- 
siderably more similar to those which ob- 
tain when extinction is introduced than 
does escape training. The higher degree 
of similarity consists essentially of two 
elements: (a) the interspersal in acquisi- 
tion of trials entirely free from shock, 
and (b) receipt of shock, when experi- 
enced at all, from an initially uncharged 
grid. The presence of these two elements 
in avoidance training and their absence 
in escape training should, through stim- 
ulus generalization, produce greater re- 
sistance to extinction among the avoid- 
ance animals than among the escape 
animals. 


= 
ESCAPE 
/ 
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Avoidance Training in Acquisition and Extinc- 
tion. 


AN ENPERIMENTAL STUDY OF EXTINCTION 


In Figure 1 the median total speeds 
for the two groups are plotted. To ob- 
tain these curves we (a) calculated the 
median total speed in each consecutive 
set of five trials for each animal, and 
(b) calculated the medians for each 
group from the individual medians for 
each 5-trial set. The group medians ap- 
pear in Figure 1 plotted at the last trial 
of the 5-trial set in question. When, in 
extinction, an animal had been discon- 
tinued, his total speed was arbitrarily 
supposed to lie below the group median 
for all 5-trial sets to follow. When the 
median animal, i.e., the sixth, extin- 
guished, the group was supposed to ex- 
tinguish and no point was plotted. 

As is apparent from Figure 1, the ex- 
tinction curve of the avoidance animals 
is considerably longer than that of the 
escape animals. Table 1 presents the sta- 
tistical evaluation of this difference in 
terms of trials to the criterion of three 


TABLE 1 


Escape vs. AVOIDANCE TRAINING: TRIALS TO 
THE CRITERION OF THREE CONSECUTIVE 
s-SECOND TRIALS 


Trials 


Group 


4 

6 
It 
14 
160 
26 
26 
20 
33 
33 
35 
37 
40 
45 
53 
55 
59 
60+ 
60+ 
60+ 
60+ 
60+ 


Meme 


= (A_ ranks) =163.0, 
ou=15.65, CR=2.33, p<.o1. 


U=24.0, U=60.5, 


consecutive trials of 5 seconds or more. 
The result shows that the difference is 
significant beyond the .o1 level. We may, 
therefore, conclude that the similarity 
theory of extinction finds confirmation 
in the comparison of escape with avoid- 
ance training. 

Does the starting or the running re- 
sponse undergo the greater deteriora- 
tion in extinction? To answer this ques- 
tion the median starting times in the 
first five and in the last five trials of ex- 
tinction were calculated for each animal. 
When an animal ran past the maximum, 
his “last five trials” were trials 56-60. The 
group medians were then calculated from 
the individual medians. In the same way 
the median running times in the first five 
and in the last five trials of extinction 
were calculated for each group. Among 
the escape animals the median starting 
time in the first five trials was 0.31 sec- 
ond and in the last five trials 0.5 second. 
This increase is very slight indeed when 
compared to the rise in running time, 
which increased from 1.54 seconds in 
the first five trials to 4.86 seconds in 
the last five. Statistics are hardly nec- 
essary: among the escape animals the ex- 
tinction effect was felt, almost entirely, 
in the running response. Even when the 
escape animal stopped altogether further 
down the grid, he still started, and with 
very nearly as much alacrity as he did 
at thé beginning of extinction. The 
avoidance animals present a very similar 
picture. Their starting times increased 
from 0.25 second to 0.38 second while 
their running times increased from 1.11 
seconds to 3.08 seconds. Once again, sta- 
tistical analysis is unnecessary. Among 
the avoidance animals, as among the es- 
cape animals, the running response was 
almost exclusively responsible for the to- 
tal extinction effect. 


Rank 
E 
E 
E 
A 
9-5 
II 
12 
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14 
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16 
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20 
20 
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20 
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ESCAPE 
— INTERMITTENT 
ESCAPE 


MEDIAN TOTAL SPEED - FEET PER SECOND 


ACQUISITION 


20 
EXTINCTION 


16 20 


Fic, 2. Median Total Speed for Escape and 
Iptermittent-Escape Training in Acquisition and 
Extinction, 


Escape vs. Intermittent-Escape Training 

The interspersal, in acquisition, of 
trials entirely free from shock is the first 
element of similarity in avoidance train- 
ing. An intermittent-escape schedule in- 
volves this element to the exclusion of 
the second, receipt of shock from an ini- 
tially uncharged grid. The present com- 
parison, therefore, enables us to examine 
in isolation the extinction effects of the 
first element of similarity in avoidance 
training. These effects, according to the 
similarity theory, should include a great- 
er resistance to extinction among the in- 
termittent-escape animals than among 
the escape animals. 

In Figure 2 the median total speeds 
for the two groups are presented. These 
curves were plotted in the same way as 
were the curves in Figure 1. Since the 
intermittent-escape animals received 
their escape training on trials 1-5, 7, 
10, 14, 19, 25, 32, and 40, grouping the 
trials by 5's destroys one of the principal 
characteristics of their acquisition curve: 
a scalloping peaked on the shock trials. 
Since, however, our present concern is 
with extinction, we will defer a detailed 


treatment of this phenomenon to the 
section on acquisition, The intermittent- 
escape group originally contained 11 ani- 
mals. Of the 11 animals, 8, those whose 
curve is plotted in Figure 2, displayed an 
increasingly stable starting-and-running 
response throughout acquisition. The 
remaining 3 animals extinguished in ac- 
quisition. This being the case, the com- 
ments on extinction which follow apply 
only to animals which responded in ac- 
quisition in one of two quite disparate 
ways. We will, when we take up acquisi- 
tion, discuss these two patterns in greater 
detail. 

Figure 2 shows a longer curve in ex- 
tinction for the intermittent-escape ani- 
mals. This difference in terms of trials 
to the criterion of three consecutive trials 
of 5 seconds or more is significant at the 
025, level (Table 2). We may, there- 
fore, safely say that the comparison of 
escape and intermittent-escape training 
lends additional support to the similarity 
theory of extinction. 

We have already seen that the escape 


TABLE 2 


Escare vs. INTERMITTENT-ESCAPE TRAINING: 
TRIALS TO THE CRITERION OF THREE 
CONSECUTIVE 5-SECOND TRIALS 


Rank 


Trials Group 


= (I ranks)=104.9, U=20.0, U=44.0, 


ou=12.11, CR=1.98, p<.025. 
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and avoidance animals extinguish almost 
exclusively in the running response. In 
this respect, intermittent-escape training 
is no different. Their starting times in- 
creased from 0.22 second to 0.26 second 
from the first five to the last five trials 
of extinction. Meanwhile, their running 
times rose from 0.92 second to 5.90 sec- 
onds. Whether we consider absolute or 
relative increase, the starting response is 
by far more resistant to extinction than 
the running response. 


Escape vs. Limited-Avoidance Training 


For the limited-avoidance animals, 
shock set in 0.8 second after they hit the 
grid. Since under this condition the rats 
could not entirely avoid shock, and since 
they received shock from an initially un- 
charged grid, the limited-avoidance 
group represents the second element of 
similarity in avoidance training, uncon- 
taminated by the first. A theory which 
implies that resistance to extinction in- 
creases with the similarity between ac- 
quisition and extinction conditions clear- 
ly requires the limited-avoidance animals 
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TABLE 3 


Escape vs. Limrrep-AVOIDANCE TRAINING: 
[RIALS TO THE CRITERION OF THREE 
CONSECUTIVE 5-SECOND TRIALS 
Trials 


Rank 


Group 


4 

6 
I! 
14 
16 
16 
20 


wn 


‘ 


‘ 


‘ 


| 
| 
| 
| 
| 
| 


ranks) =164.0, U=23.0, U =60.5, 


CR=2.40, p<.or. 


to run longer in extinction than the es- 
cape animals. 

In Figure 3 the acquisition and ex- 
tinction curves are plotted in the same 
way as in the first two figures. 

In terms of trials to the criterion of 
three consecutive trials of 5 seconds or 
more, the similarity theory of extinction 
is confirmed (Table 3). The limited- 
avoidance animals ran longer in extinc- 
tion than did the escape animals. Sig- 
nificance reaches the .o1 level. 

All three groups examined so far have 
shown the effects of extinction almost 
entirely in the running response. The 
limited-avoidance animals complete the 
picture. Their median starting times in- 
creased from 0.31 second to 0.42 second 
over the extinction period. Meanwhile, 
their median running times increased 
from 1.16 seconds to 4.74 seconds. In all 
four groups, then, the starting response 
was decidedly more resistant to extinc- 
tion than was the running response. 


7 
I 
| 
20 
32 9 
33 10 
35 
37 12 | 
13 
55 | 14 
60 18. | 
60 18. 
60 is. 
60 18. | 
60 18. 
60 18. 
60 18. 
60 18. 
| 
3 wee 
dj \ 
2 ! \A\ 
\ 
10 
TRIALS 


MARSHALL B. JONES 


V. RECOVERY 


Escape and Intermittent-Escape Train- 
ing 

a. The starting response. Estes in his 
experimental study of punishment (2) 
presented a wide variety of results in 
support of the hypothesis that a response 
which has been continuously or intermit- 
tently punished will, if performed and 
not punished, recover in strength. At 
first blush, this finding might seem of 
little relevance to the present experi- 
ment. On closer analysis, however, recov- 
ery is seen to play a crucial role in the 
behavioral systems under investigation. 

Since the words “punishment” and 
“punished” figure prominently our 
analysis, a brief account of the way we 
use them may be in order. The word 
“punishment” applies to stimuli, like 
shock, which, quite apart from learning, 
produce in an animal the usual signs of 
fear: defecation, urination, quivering, 
squealing, etc. The word “punished,” on 


the other hand, applies to responses. 


Whether or not a response is punished 
depends upon the difference in punish- 
ment before and alter the response. The 
simple fact that punishment follows a 
response does not mean that the response 
is punished. A response is punished only 
if its performance results in “a turn for 
the worse.” 

With this terminological understand- 
ing we turn to the starting response 
among the escape intermittent- 
escape animals. More specifically, we 
are concerned with the shock trials in the 
schedules of these two groups. Typically, 
the animals of all four groups landed on 
the grid with their hind legs and leaped 
forward. If the two hind legs fell on 
even-spaced spokes, they drew no cur- 
rent and the animal did not receive shock 
until after his initial thrust had already 


carried him beyond the first beam. If 
the two hind legs fell on odd-spaced 
spokes, they drew current, but the shock 
experienced was surely less than that 
which obtained after the starting re- 
sponse had been completed and all four 
legs were in contact with the grid. In 
short, the starting response among the es- 
cape and intermittent-escape animals 
was punished in every shock trial. On 
the nonshock trials punishment neither 
preceded nor followed the starting re- 
sponse. Hence, the starting response 
among the escape and intermittent- 
escape animals went unpunished in their 
nonshock trials. 

Since for the escape animals, every 
acquisition trial was a shock trial, the 
starting response among the escape ani- 
mals was punished in every trial up to 
and including the goth. From the 41st 
trial on, the starting response was no 
longer punished. Hence, it Estes’ conclu- 
sions hold generally, the starting  re- 
sponse among the escape animals should, 
beginning with the 41st trial, gradually 
recover in strength, i.e., the correspond- 
ing starting times should gradually de- 
crease. 

The starting response among the in- 
termittent-escape animals was, as we 
have seen, punished in trials 1-5, 7, 10, 
14, 19, 25, 32, and 4o, and unpunished 
in trials 6, 8-9, 11-13, 15-18, 20-24, 
26-91, and 33~39. Let us call these latter 
series of 1, 2, 3, 4, 5, 6, and 7 trials 
“null” series. Clearly, each null series 
follows a series of trials in which the 
starting response has been continuously 
or intermittently punished, We should, 
therefore, expect to find recovery of the 
starting response in the null series of the 
acquisition schedule. More concretely, 
starting times should tend to decrease 
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during the null series of intermittent- 
escape Waining. 

If recovery of the starting response is 
manifest throughout acquisition under 
intermittent-escape training but delayed 
under escape training until the 41st trial, 
then the starting response among the 
intermittent-escape animals should, to- 
ward the end of acquisition, be stronger, 
i.e., starting times should be lower than 
they are among the escape animals. 

To test the first consequence of the 
foregoing analysis we compared the me- 
dian starting time (Mdn,) in the first five 
trials of extinction with the median 
starting time (Mdn,) in the second five 
trials of extinction for each escape ani- 
mal. If recovery is indeed present, the 
medians Mdn, should be less than the 
medians Mdn,. This is, in fact, the case. 
Significance reaches beyond the .o1 level 
(Table 4). 

To test the second consequence, we 
calculated for each intermittent-escape 
animal the median starting time in the 
first and in the second halves of each 
null series, with the obvious exception of 
the first. From these we calculated, again 
for each animal, the median Mdn, of 
the first half medians and the median 
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INTERMITTENT-ESCAPE TRAINING: MEDIAN 
STARTING TIMES IN THE FIRST HALF VS. THE 
Seconp HALF OF THE NULL SERIES 
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Mdn, of the second half medians. If re- 
covery occurs in the null series of the 
intermittent-escape schedule, the Mdn, 
medians should be less than the Mdn, me- 
dians. The results bear out the analysis. 
Significance reaches beyond the .os5 level 
(Table 5). In 6 out of 48 null series at 
least one starting time was lost; the rat 
jumped the beam. Since the measure em- 
ployed is very delicate, these series were 
thrown out in preference to estimating 
the medians from the trials that re- 
mained. 

To test the third consequence of our 
analysis, we Compared the median start- 
ing times of the escape animals with 
those of the intermittent-escape animals 
in the last five trials of acquisition. The 
intermittent-escape animals definitely 
started faster. The result is significant 
at the .o1 level (Table 6). 

In Figure 4 median starting speeds 
in acquisition and the first ten trials of 
extinction are plotted for the escape 
and intermittent-escape groups. ‘The 
curves are plotted in sets of five trials 
as were the earlier curves for total 
speed. In Figure 4 the first and third 
consequences of our analysis may be seen 
graphically. 

In explaining Fstes’ results we have 
argued in another place (7) that an ani- 
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TABLE 6 
Escape vs. INTERMITTENT-ESCAPE TRAINING: 
MEDIAN STARTING TIMES IN THE Last 
Five TRIALS oF ACQUISITION 


Seconds 


Rank 


Group 


ranks)=146.5, U=7.5, U=44.0, 
Oy=12.11, CR=3.01, p<.or. 


mal which has been punished, continu- 
ously or intermittently, for performing 
a given response in a particular situation 
acquires in consequence an expectation 
that the response in question will be 
punished when performed in that situa- 
tion. If, having acquired such an expec- 
tation, the animal performs the response 
and is not punished, he is effectively re- 
warded. As a result, the response recovers 
in strength. This view of recovery is 
clearly adequate to the present results as 
well as to Estes’ original findings. ‘The 
escape animals, for example, build up in 
acquisition an expectation that they will 
be punished for starting. When, in ex- 
tinction, they start and are not punished, 
their starting response is effectively re- 
warded. In consequence, their starting 
response recovers in strength, their start- 
ing times gradually decrease. An alto- 
gether similar argument applies to recov- 
ery in the null series of the intermiitent- 
escape schedule. ‘The third consequence 
of our analysis follows, as we have seen, 
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from the first two and, hence, from the 
interpretation of recovery as the result 
of effective reward. 

b. The running response, The escape 
and intermittent-escape animals experi- 
enced two sorts of trials: shock trials in 
which the animals landed on a perma- 
nently charged grid, and nonshock trials 
in which the grid was never charged. In 
the first case shock preceded but did not 
follow the running response. In the sec- 
ond case shock neither preceded nor fol- 
lowed the running response. In neither 
case, then, was the running response 
punished. Hence, the running response 


o——o ESCAPE 
— INTERMITTENT 
ESCAPE 


ANDTIN( 


ACQUISITION EXTINCTION 


Fic. 4. Median Starting Speed for Escape and 
Intermittent-Escape Training in Acquisition and 
the First Ten Trials of Extinction. 
could hardly show recovery and, as a 
matter of fact, never did. 

The presence of recovery in the start- 
ing response and its total absence in the 
running response provide, in our opin- 
ion, an explanation of the earlier finding 
that the starting response in these two 
groups was decidedly more resistant to 
extinction than was the running re- 
sponse, In “extinction” the starting re- 
sponse actually increased its strength, 
at least for awhile. In contrast, the run- 
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ning response, unbolstered by recovery, 
deteriorated from the outset of the ex- 
tinction schedule. 


Avoidance .and  Limited-Avoidance 
Training 

a. The starting response. The start- 
ing response among the limited-avoid- 
ance animals, just as among the escape 
animals, was punished in every acquisi- 
tion trial. In this respect the sole dif- 
ference between the two groups was that 
the starting response received shock im- 
mediately among the escape animals but 
only after a delay of approximately 0.5 
second among the limited-avoidance ani- 
mals. Among the avoidance animals the 
starting response was also punished at 
a delay. However, in many, if not most, 
of their acquisition trials the avoidance 
animals avoided shock altogether, and 
the starting response went unpunished. 
In neither group was the starting re- 
sponse punished under the extinction 
schedule. In brief, then, the two avoid- 
ance groups were, with respect to re- 
covery, altogether analogous to the two 
escape groups, the delay factor, of course, 
excepted, 

By virtue of the analogy, three parallel 
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TABLE 8 
AVOIDANCE vs. LIMITED-AVOIDANCE TRAINING: 
MEDIAN STARTING TIMES IN THE LAST 
Five TRIALS OF ACQUISITION 
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consequences follow: (a) the starting re- 
sponse among the limited-avoidance ani- 
mals should show recovery in the first ten 
trials of extinction, (b) the starting re- 
sponse among the avoidance animals 
should show recovery during the null se- 
ries of their acquisition schedule, ie., 
during those trials in which they avoided 
shock altogether, and (c) the starting 
response among the avoidance animals 
should, toward the end of acquisition, be 
stronger than among the limited-avoid- 
ance animals. 

To test the first consequence we com- 
pared median starting times among the 
limited-avoidance animals in the first 
five and in the second five trials of ex- 
tinction. ‘The result confirms the pres- 
ence of recovery at the .o5 level (Table 
7): 

We have made every effort to find an 
adequate measure of recovery in the ac- 
quisition series of the avoidance animals. 
However, the extreme irregularity in 
shock pattern from animal to animal 
frustrated our every attempt. A direct 
approach to recovery in acquisition 
among the avoidance animals has, there- 
fore, been abandoned. 

We may, however, approach the prob- 
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lem indirectly, The third consequence 
ol our analysis follows, as we have seen, 
from the first two. Since the first conse- 
gence was independently confirmed, 
direct confirmation of the third consti- 
tutes at the same time indirect support 
for the second. Comparing median start- 
ing times in the last five trials of acqui- 
sition, we found that the avoidance ani- 
mals did indeed start faster than the 
limited-avoidance animals. Significance 
reaches beyond the .04 level (Table 8). 
This result confirms the third conse- 
quence and, hence, the second indirectly. 
In Figure 5 median starting speeds in 
acquisition and the first ten trials of ex- 
tinction are plotted for the avoidance 
and limited-avoidance animals. ‘The 
role of recovery may be clearly seen. 
The results of the last section fell quite 
naturally under the rubric of recovery as 
the result of eflective reward. The re- 
sults of this section may be analyzed in 
almost the same words. The starting re- 
sponse among the limited-avoidance ani- 
mals was punished at a delay through- 
out acquisition. When, in extinction, the 
limited-avoidance animal starts and is 
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Fic, 5. Median Starting Speed for Avoidance 
and Limited-Avoidance Training in Acquisition 
and the First Ten Trials of Extinction. 


not punished, starting is effectively re- 
warded. In consequence, the starting re- 
sponse recovers in strength. A null series 
of avoidance training, Le., a series of 
trials in which the animal successfully 
avoids shock, always follows a series of 
trials in which the starting response has 
been continuously or intermittently 
punished at a delay. When the avoidance 
animal starts and is not punished (be- 
cause he successtully avoids), the starting 
response is effectively rewarded. Recov- 
ery follows as a result. The third conse- 
quence follows from the first two and, 
hence, from our view of recovery, as the 
result of effective reward. 

b. The running response, In the last 
section we saw that neither escape sched- 
ule allowed the running response to be 
punished. In the two avoidance groups, 
on the other hand, the possibility exists 
that the running response may be fol- 
lowed and not preceded by shock. The 
limited-avoidance animals, for example, 
might receive the onset of shock while 
in a full run toward the goal box. Were 
this to take place with any consistency, 
we would expect to find evidence of re- 
covery in the running response when 
shock was removed. However, though the 
possibility exists, the running response 
was not consistently punished in either 
avoidance group. 

Each individual limited-avoidance ani- 
mal tended in acquisition to a fairly 
constant total starting-and-running time. 
In consequence, he received the onset of 
shock from trial to trial at approximately 
the same point along the grid. The typi- 
cal result was a pause just before the 
point where shock usually set in. This 
tendency, as we shall see in the next sec- 
tion, was sufficiently powerful to pro- 
duce a significant increase in running 
time for the limited-avoidance animals 
toward the end of acquisition. In any 
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case, the running response, at least in the 
later stages of acquisition, was not con- 
sistently punished. Pausing, however, 
was. In consequence, removal of shock, so 
far from producing recovery in the run- 
ning response, should have produced re- 
covery in the pausing response, i.€., con- 
firmed the animals in their tendency to 
pause toward the end of the grid. What 
has been said of the limited-avoidance 
animals applies to the avoidance animals 
as well. They too slowed-up or stoppe:' 
when shock set in, particularly in’ the 
later stages of acquisition. In conse- 
quence, slowing-up, not running, was 
confirmed by recovery when shock was 
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omitted. Hence, as we should have ex- 
pected, recovery of the running response 
was altogether absent among the animals 
of the two avoidance groups. 

These observations enable us to under- 
stand the fact, previously reported, that 
the starting response among the animals 
of the two avoidance groups was dis- 
tinctly more resistant to extinction than 
was the running response. The starting 
response in these two groups, just as in 
the two escape groups, was preserved and 
strengthened by recovery, while the run- 
ning response was not. In consequence, 
the running response was less able to 
resist extinction and deteriorated first. 


VI. ACQUISITION 


Avoidance and Limited-Avoidance 
Training 

Shetheld and Temmer reported that 
their avoidance animals showed a tend- 
ency to slow up in the later stages of 
acquisition. A glance at Figure 1 reveals 
that our animals exhibited a similar tend- 
ency. The median total time increased 
from 1.91 seconds in trials 16-20 to 1.43 
seconds in trials 36-40. In terms of com- 
ponents, median running times increased 
from 1.02 to 1.18 seconds, while median 
starting times decreased from 0.30 to 0.27 
second for the same sets of trials. The 
slowing-up was, therefore, a phenomenon 
confined to the running response. ‘Table 
g reveals the increase in running time as 
significant at the .o5 level. To under- 
stand the result we need only note that 
the avoidance animal typically started 
avoiding around the 1oth trial, For some 
time afterwards, his running times re- 
mained very low. Then they started to 
increase, as a simple matter of extinction, 
until toward the end of acquisition he 
picked up a few more shocks. The ani- 
mals varied, of course, as to when they 


started avoiding and how long they con- 
tinued to avoid once they had started, 
(On the average, however, their running 
times were lowest in trials 16-20 and 
highest, from then on in acquisition, in 
trials 36-40.) This pattern of shock and 
avoidance accounts in the main for the 
result reported above. 

Largely as a consequence of the first 
result, Shefheld and Temmer reported 


that the escape animals were running 
faster toward the end of acquisition than 
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the avoidance animals. Once again Fig- 
ure 1 reveals a similar difference between 
our two groups. In the last five trials of 
acquisition the total times for the escape 
and avoidance groups were 1.24 and 1.43 
seconds respectively. As we would ex- 
pect, the difference is one of running 
time exclusively. The starting times of 
the avoidance animals were lower than 
those of the escape animals. More pre- 
cisely, the median starting time in trials 
36-40 for the avoidance animals was 0.27 
second and for the escape animals 0.32 
second, The running times, however, 
were definitely lower for the escape ani- 
mals than for the avoidance animals, be- 
ing 0.85 and 1.18 seconds respectively. 
The significance of the difference reaches 
beyond the .o2 level (Table 10). 

As mentioned in the last section, the 
limited-avoidance animals showed a tend- 
ency to pause just before their usual 
shock point. This tendency produced the 
falling-off in total speed which can be 
observed in Figure 3. As is to be ex- 
pected, this falling-off occurred solely in 
the running response. The median start- 
ing times in trials 16-20 and in trials 
36-4o were equal at 0.33 second. Mean- 
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while, median running time increased 
from 0.91 to 1.05 seconds for the same 5- 
trial sets. This increase is significant at 
the .o25 level (Table 11). 


Escape and Intermittent-Escape Training 

The acquisition data of the escape ani- 
mals contain no result of theoretical im- 
portance which has not already been pre- 
sented, 

The intermittent-escape group, as 
mentioned earlier, originally numbered 
11 animals. The results reported as in- 
termittent escape were based on 8 of 
the original 11 animals. These 8 animals, 
as well as the remaining g, constituted a 
very group. The 
groups of intermittent-escape animals 
may be distinguished one from the other 
on any one of a number of criteria. The 
most obvious, perhaps, is the following: 
any animal which showed a criterion 
trial, i.e., a trial of 5 seconds or more, be- 
tween trials 20 and 40 belonged to the 
group of 3; and any animal which did 
not, to the group of 8. Everything we have 
said, therefore, about intermittent-escape 
animals applies only to animals who did 
not give a criterion trial between trials 20 
and 4o. Needless to say, no animal of 
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TABLE 12 
INTERMITTENT-ESCAPE TRAINING: NUMBER OF 
10-SECOND TRIALS IN THE First Five 

TRIALS OF ACQUISITION 
vs. Group-of- -8) 


Trials Group 


O 


(3 ranks) = 28.5, U=1.5, p<.o2. 


the 44 used, excepting these three, gave 
a criterion trial in the second half of the 
acquisition series. 

While three animals are far too few to justify 
much theory, they may be enough to warrant 


a few comments. Early shock, i.e., shock in the 
first few moments affer dropping, can become 


conditioned to the starting-and-running response. 


Once early shock calls out the response, the asso- 
ciates of early shock, dropping and the feel of 
the grid, can also come to call out the response. 
In the first five trials of acquisition, conditioning 
of the starting-and-running response is subject 
to considerable chance variation. A given animal 
may take two or three more trials than the 
average to try running to the goal box. If he 
does, neither early shock nor its associates will 
have become conditioned by the sixth trial. In 
consequence, the rat enters the intermittent 
schedule with a poorly developed, perhaps non- 
existent, starting-and-running response. As a re- 
sult, the response never does develop and the 
rat is simply “left behind.” In support of this 
interpretation we note that the group of § 
gave 2, 3, and 4 full 10-second trials in the first 
five trials of acquisition, while none of the group 
of 8 gave more than 2 such trials in the same 
interval. Table 12 reveals this difference as sig- 
nificant at the .o2 level. This result serves to con- 
firm the hypothesis that the group of 3 repre- 
sents faulty development of the starting-and- 
running response in the first five trials of acquisi- 
tion. Finally, we should like to note that these 
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variations in early development of the starting- 
and-running response did not play any appre- 
ciable role under the other three schedules. In 
all three the asymptotes for the response were 
reached long before the acquisition series was 
ended. Only under the intermittent schedule 
could faulty early development have had far- 
reaching effects. 


The group of 8, as was also mentioned 
earlier, showed a definite scalloping in 
acquisition. In Figure 6 the curve for this 
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group is plotted for each acquisition 
trial in terms of median total speed. The 
curve shows one remarkably consistent 
attribute, its duality. The curve consists 
actually of two curves, one for the shock 
trials and one for the nonshock trials. 
Both have the familiar logarithmic form. 
They differ, principally, in their asymp- 
totes, the shock curve being considerably 
higher than the nonshock. Each shock 
trial was not only faster than the last but 
appeared to add an increment of speed 
to the nonshock trials to follow. 
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Let us summarize the theory and re- 
sults which have been our concern in the 
preceding pages. Out of the conflict be- 
tween intermittent reinforcement and 
early formulations of the law of effect 
emerged a theory of extinction. ‘This the- 
ory, reduced to testable essentials, is that 
resistance to extinction increases with 
the similarity between acquisition and 
extinction conditions. While this hy- 
pothesis had been widely checked in out- 
right studies of intermittent reinforce- 
ment, its consequences for escape and 
avoidance schedules had gone unexam- 
ined. More specifically, the acquisition 
conditions of avoidance, intermittent- 
escape, and limited-avoidance training 
are all more similar to extinction condi- 
tions than are the acquisition condi- 
tions of escape training. Hence, the simi- 
larity theory of extinction entails three 
definite consequences: avoidance, inter- 
mittent-escape, and limited-avoidance 
training should all prove to be more re- 
sistant to extinction than escape train- 
ing. All three consequences were con- 
firmed by the experimental results. 

The total response of reaching the 
goal box is analyzable into two com- 
ponents, a starting and a running re- 
sponse. In all four groups the extinction 
effect was felt almost exclusively in the 
running response. ‘The starting response 
remained virtually intact long after the 
running response had almost entirely dis- 
appeared. An explanation of this finding 
was delayed until after the analysis of 
recovery had been presented. 

An analysis of the starting response 
among the escape and intermittent- 
escape animals led to three definite ex- 
pectations, all of which were supported 
by the evidence: 

1. The starting response among the 
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VII. CONCLUSION 


escape animals should show recovery in 
the early stages of extinction. 

2. The starting response among the 
intermittent-escape animals should show 
recovery during the null series of their 
acquisition schedule, 

3. intermittent-escape animals 
should, in the later stages of acquisition, 
start faster than the escape animals. 

Since the running response among 
the escape and intermittent-escape ani- 
mals was never punished, there could be 
no recovery of the running response after 
punishment. The facts of recovery in 
the starting response and no recovery in 
the running response were offered as an 
explanation of the earlier result that the 
starting response was far more resistant 
to extinction than the running response 
in these two groups of animals. 

The starting respgnse among the 
avoidance and limited-avoidance animals 
was then subjected to an analysis alto- 
gether analogous to that which had been 
applied earlier to the starting response 
among the escape and _ intermittent- 
escape animals. The analysis issued in 
three analogous expectations: 

1. The starting response among the 
limited-avoidance animals should show 
recovery in the early stages of extinction. 

2. The starting response among the 
avoidance animals should show evidence 
of recovery during the null series of their 
acquisition schedule. 

3. The starting response among the 
avoidance animals should be, toward the 
end of acquisition, faster than among the 
limited-avoidance animals. 

Because of the irregularity of shock 
trials among the avoidance animals no 
adequate measure of recovery during 
their acquisition series could be found. 
As a result, consequence 2 could not be 
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tested directly. The remaining two con- 
sequences were, however, confirmed. 

The running response among the 
avoidance and limited-avoidance animals 
was rarely punished. In general, the ani- 
mals were slowing up or stopping when 
shock set in, particularly in the later 
stages of acquisition. In consequence, 
recovery, so far from strengthening the 
running response, strengthened the slow- 
ing-up and stopping responses. 

The absence of recovery in the run- 
ning response and the presence of recov- 
ery in the starting response among the 
avoidance and limited-avoidance animals 
were offered in explanation of the de- 
cidedly greater resistance to extinction 
of the starting response among the ani- 
mals of these two groups. 

The results involving recovery were 
seen to follow from the interpretation of 
recovery as the result of effective reward. 
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The running response of the avoid- 
ance animals tended to slow up toward 
the end of acquisition. Largely in con- 
sequence of this result, the escape ani- 
mals were running faster than the avoid- 
ance animals in the later stages of ac- 
quisition, The tendency of the limited- 
avoidance animals to pause just before 
their usual shock point produced a sig- 
nificant slowing-up in their running re- 
sponse toward the end of acquisition. 

The group of three intermittent-escape 
animals which “extinguished in acquisi- 
tion” was seen to have been a group in 
which conditioning of the starting-and- 
running response was faulty in the first 
five trials of acquisition. Finally, the 
dominant group of eight displayed two 
distinct acquisition curves: one for the 
shock trials and one for the nonshock 
trials. 
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